Computer aided diagnosis (cad) apparatus and method

ABSTRACT

Disclosed are Computer Aided Diagnosis (CAD) apparatus and method to combine information on sequential image frames and to provide a superior classification result for the ROI in the image frame. The CAD apparatus may include a Region of Interest (ROI) detector configured to detect an ROI from image frames, a categorizer configured to create groups of image frames having successive ROI sections from among the image frames based on a result of the detection, a classifier configured to classify an ROI detected from each of the image frames belonging to the groups, and a result combiner configured to combine classification results for the image frames belonging to each group from the groups and to calculate a group result for the each group.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/138,405, filed on Sep. 21, 2018, which is a continuation of priorapplication Ser. No. 14/950,543, filed on Nov. 24, 2015, which hasissued as U.S. Pat. No. 10,083,502 on Sep. 25, 2018, and was based onand claimed priority under 35 U.S.C. § 119(a) of a Korean patentapplication number 10-2014-0166736, filed on Nov. 26, 2014, in theKorean Intellectual Property Office, the entire disclosure of each ofwhich is incorporated herein by reference for all purposes.

BACKGROUND 1. Field

The following description relates to a Computer Aided Diagnosis (CAD)technology.

2. Description of Related Art

A Computer Aided Diagnosis (CAD) assists medical professionals byanalyzing medical images, i.e., ultrasonic images, and marking anabnormal area in each medical image based on the analysis. Because oftheir limited perception capabilities, it is difficult for humans toperform such diagnosis without errors. In addition, analyzing eachmedical image requires a large amount of time, great attention, andcare. To address these drawbacks, the CAD system is designed to improveaccuracy of diagnosis and alleviate the burden that medicalprofessionals may feel during analysis.

Since the CAD system classifies each region of interest (ROI) detectedfrom a different image frame, different classification results may beobtained regarding image frames of the same lesion. There is need for atechnology that combines information on all image frames for each lesionand provides a single classification result for each lesion.

SUMMARY

In one general aspect, there is provided a Computer Aided Diagnosis(CAD) including a Region of Interest (ROI) detector configured to detectan ROI from image frames, a categorizer configured to create groups ofimage frames having successive ROI sections from among the image framesbased on a result of the detection, a classifier configured to classifyan ROI detected from each of the image frames belonging to the groups,and a result combiner configured to combine classification results forthe image frames belonging to each group from the groups and tocalculate a group result for the each group.

The categorizer may be further configured to group the plurality ofimage frames into a group based on an image frame where no ROI isdetected.

A classification result obtained by the classifier may include at leastone of BI-RADS Lexicon information or BI-RADS Category information on acorresponding ROI.

The result combiner may be further configured to calculate a score foreach BI-RADS Lexicon item or each BI-RADS Category item in the eachgroup based on the classification results obtained by the classifier,and to select a BI-RADS Lexicon item with the highest score or a BI-RADSCategory item with the highest score as a group result.

The result combiner may be further configured to calculate the score foreach BI-RADS Lexicon item or each BI-RADS Category item in the eachgroup by adding up frame scores of image frames corresponding to eachBI-RADS Lexicon item or each BI-RADS Category item in the group, andwherein the frame scores may represent weights assigned to each of theimage frames.

The frame score may be calculated based on a size of an ROI detectedfrom each of the image frames in the each group and a model fitness ofthe each of the image frames in the each group.

The result combiner may be further configured to generate a histogram ofthe group based on the calculated score for each BI-RADS Lexicon item oreach BI-RADS Category item in the each group.

The result combiner may be further configured to determine whether anROI of the each group is malignant or benign based on the BI-RADSLexicon item with the highest score or the BI-RADS Category item withthe highest score in the each group.

The CAD apparatus may include a similar image combiner configured tocombine image frames satisfying a condition from among continuous imageframes, and to select any one of the combined images as a representativeimage frame, wherein the result combiner may be further configured touse the representative image frame when combining the classificationresults.

The condition may include at least one of a condition where a change insize of an ROI is smaller than a threshold value and a condition where amaximum size of an ROI is smaller than a threshold size.

The CAD apparatus may include a similar group combiner configured togenerate a histogram representing characteristics of an ROI of the eachgroup, and to combine similar groups into one group using a histogramclustering algorithm.

The histogram clustering algorithm may include one of a HierarchicalAgglomerative Clustering algorithm and a Bhattacharyya Coefficientalgorithm.

The CAD apparatus may include a display component configured to displaythe group classification result for the group on a screen.

The CAD apparatus may include a feature value extractor configured toextract a feature value from contour of the ROI detected from imageframes, an area adjacent to the contour, or pixels inside the contour,and the classifier may be further configured to classify the ROIdetected in each image frame based on comparing a feature valueextracted from the ROI with a pre-stored diagnostic model.

In another general aspect, there is provided a Computer Aided Diagnosis(CAD) method including a processor performing operations of detecting aRegion of Interest (ROI) from a plurality of image frames, groupingimage frames having successive ROI sections from among the plurality ofimage frames based on a result of the detection, classifying an ROIdetected from each of the image frames belonging to a group, andcalculating a group result for the group by combining classificationresults for the image frames belonging to the group.

The grouping of image frames may include grouping the plurality of imageframes with reference to an image frame where no ROI is detected.

The classifying of the ROI may include identifying at least one ofBI-RADS Lexicon information or BI-RADS Category information on the ROIdetected from each of the image frames.

The calculating of the group result may include calculating a score foreach BI-RADS Lexicon item or each BI-RADS Category item in the imageframes belonging to the group, and selecting a BI-RADS Lexicon item withthe highest score or BI-RADS Category item with the highest score in thegroup as the result of classification for the group.

The calculating of the score for each BI-RADS Lexicon item or eachBI-RADS Category item in the group may include calculating a frame scorebased on size of an ROI detected from each of the image frames and modelfitness of each of the image frames, and calculating the score for eachBI-RADS Lexicon item or each BI-RADS Category item in the group byadding up frame scores for image frames in the group.

The CAD method may include generating a histogram of the group based onthe calculated score for each BI-RADS Lexicon item or each BI-RADSCategory item in the group.

The CAD method may include combining image frames that satisfies acondition from among continuous image frames, and selecting any one ofthe combined image frames as a representative image frame, using therepresentative image frame in combining the classification results forthe image frames belonging to the group, and wherein the condition mayinclude at least one of a condition where a change in size of an ROI isless than a threshold value and a condition where a maximum size of anROI is less than a threshold size.

The CAD method may include generating a histogram representingcharacteristics of an ROI of each group, and combining similar groupsinto one group using a histogram clustering algorithm.

The model fitness of the each of the image frames may correspond to aprobability distribution of the each BI-RADS Lexicon item or eachBI-RADS Category item.

In another general aspect, there is provided a Computer Aided Diagnosis(CAD) method including a processor performing operations of detecting aRegion of Interest (ROI) from a plurality of image frames, groupingimage frames having successive ROI sections from among the plurality ofimage frames based on an image frame where no ROI is detected,classifying an ROI detected from each of the image frames belonging to agroup based on a feature of the ROI, and calculating a group result bycombining classification results for the image frames belonging to thegroup.

The feature may be extracted from contour of the ROI detected from imageframes, an area adjacent to the contour, or pixels inside the contour.

Other features and aspects may be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a Computer AidedDiagnosis (CAD) apparatus.

FIG. 2 is a diagram illustrating another example of a CAD apparatus.

FIGS. 3A and 3B are diagrams illustrating examples for explainingdetailed operations of a CAD apparatus.

FIG. 4 is a diagram illustrating an example of a final classificationresult on a screen.

FIG. 5 is a diagram illustrating an example of a CAD method.

FIG. 6 is a diagram illustrating another example of a CAD method.

Throughout the drawings and the detailed description, unless otherwisedescribed, the same drawing reference numerals will be understood torefer to the same elements, features, and structures. The relative sizeand depiction of these elements may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the systems, apparatuses, and/ormethods described herein will be apparent to one of ordinary skill inthe art. The progression of processing steps and/or operations describedis an example; however, the sequence of and/or operations is not limitedto that set forth herein and may be changed as is known in the art, withthe exception of steps and/or operations necessarily occurring in acertain order. Also, descriptions of functions and constructions thatare well known to one of ordinary skill in the art may be omitted forincreased clarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided so thatthis disclosure will be thorough and complete, and will convey the fullscope of the disclosure to one of ordinary skill in the art.

FIG. 1 is a diagram illustrating an example of a Computer AidedDiagnosis (CAD) apparatus. Referring to FIG. 1, a CAD apparatus 100includes a region of interest (ROI) detector 110, a grouping component120 (may also be referred to as a categorizer), a classifier 130, and aresult combiner 140.

The ROI detector 110 may detect ROIs from a plurality of image framesusing a lesion detection algorithm. An ROI may include not only amalignant lesion area, but also a lesion area that is not yet identifiedas either malignant or benign and a lesion area having unique features.The lesion detection algorithm may include algorithms such as, forexample, AdaBoost, Deformable Part Models (DPM), Deep Neural Network(DNN), Convolutional Neural Network (CNN), and Sparse Coding. Theplurality of image frames forms a sequence of continuously capturedimage frames, and may include images captured using equipment such as,for example, Computed Radiography (CR), Computed Tomography (CT),ultrasonic images, Magnetic Resonance Image (MRI), and Positron EmissionTomography (PET).

The grouping component 120 may group image frames having successive ROIsections from among a plurality of image frames into a group based on aresult of ROI detection.

In an example, the grouping component 120 may group a plurality of imageframes into different groups with reference to an image frame in whichno ROI is detected. For example, among image frames 1 to 10, ROI A isdetected from the image frames 1 to 4, ROI B is detected from the imageframes 7 to 10, and no ROI is detected from the image frames 5 and 6.The grouping component 120 groups the image frames 1 to 4 with ROI Ainto group A, and the image frames 7 to 10 with ROI B into group B. ROIA and ROI B may be identical or different. The grouping component 120may group image frames of the same ROI into different groups accordingto whether the ROI is continuously detected.

The classifier 130 may classify an ROI detected from an image frame ofeach group. According to an example, the classifier 130 may extract afeature value from an ROI detected from an image frame and classify theROI based on the extracted feature value. The classifier 130 may includea feature value extractor 131 and an ROI classifier 132.

The feature value extractor 131 may extract a feature value from contourof an ROI detected from an image frame, an area adjacent to the contour,or information on pixels in the area inside the contour. A featureindicates a characteristic used to determine whether a correspondingarea is a lesion area, and a feature value is a numeric valuerepresenting the feature. For example, in the case of breast imaging,according to Breast Imaging Reporting And Data System (BI-RADS) Lexiconclassification a feature may be a lesion feature, such as, for example,shape, margin, echo pattern, orientation, boundary.

The feature value extractor 131 may be implemented using various imagerecognizing and machine learning algorithms, such as, for example,Deformable Part Model (DPM), Diabolo network, and Auto Encoder.

The ROI classifier 132 may classify an ROI in each image frame bycomparing a feature value extracted from the ROI with a pre-storeddiagnostic model. In a non-exhaustive example, a classification resultfor the ROI may include information such as, for example, BI-RADSLexicon information, BI-RADS category information, and whether the ROIis malignant/benign.

The diagnostic model may be generated through machine learning usingfeature values extracted from a pre-collected diagnostic images. Thegenerated diagnostic model may be stored in a database included in thedeterminer 130 or may be stored in an external database.

The machine learning algorithm may include algorithms such as, forexample, artificial neural network, decision tree, Genetic Algorithm(GA), Genetic Programming (GP), Gauss Process Regression, LinearDiscriminant Analysis, K-Nearest Neighbor(K-NN), perceptron, RadialBasis Function Network (RBFN), Support Vector Machine (SVM), anddeep-learning.

The result combiner 140 may generate a group classification result for agroup by combining classification results for image frames belonging tothe group.

The result combiner 140 may calculate a score for each BI-RADS Lexiconitem or each BI-RADS Category item in a group based on BI-RADS Lexiconinformation or BI-RADS Category information on an ROI detected from eachimage frame belonging to the group. The result combiner 140 may select aBI-RADS Lexicon item with the highest score or a BI-RADS Category itemwith the highest score as a group classification result for the group.

According to an example, a score for a BI-RADS Lexicon item or a BI-RADSCategory item in a group may be the number of image frames correspondingto the BI-RADS Lexicon item or the BI-RADS Category item in the group.That is, the result combiner 140 may calculate a score for each BI-RADSLexicon item or each BI-RADS Category item in the group by calculatingthe number of image frames corresponding to each BI-RADS Lexicon item oreach BI-RADS Category item in the group.

For example, when group A consists of image frames 1 to 10, and imageframes 1 and 10 are classified as BI-RADS Category 2, the image frames 2and 9 are classified as BI-RADS Category 3, the image frames 3 to 6 areclassified as BI-RADS Category 4A, and the image frames 7 and 8 areclassified as BI-RADS Category 4B. The result combiner 140 may calculatethe number of image frames corresponding to a BI-RADS Category item tobe a score for the BI-RADS Category item. Thus, with respect to an ROIof the group A, the result combiner 140 may calculate a score of BI-RADSCategory 2 to be 2, a score of BI-RADS Category 3 to be 2, a score forBI-RADS Category 4A to be 4, and a score for BI-RADS Category 4B to be2.

According to another example, a score for an item may be defined as asum of frame scores of image frames corresponding to the BI-RADS Lexiconitem or the BI-RADS Category item, wherein the frame scores representweights assigned to the image frames. The result combiner 140 maycalculate a score for each BI-RADS Lexicon item or each BI-RADS Categoryitem in a group by adding up weighted frame scores of image framescorresponding to each BI-RADS Lexicon item or each BI-RADS Category itemin the group.

For example, when a frame score of each of the frames images 1, 2, 9,and 10 is 0.1, that a frame score of each of the image frames 3 and 8 is0.3, and that a frame score of each of the image frames 4 to 7 is 0.5.In this case, the result combiner 140 calculate a score for each item byadding up frame scores of image frames corresponding to each BI-RADSCategory item. With respect to the ROI of group A discussed above, theresult combiner 140 may calculate a score for BI-RADS Category 2 to be0.2, which is a sum of 0.1 (a frame score of the image frame 1) and 0.1(a frame score of the image frame 10). The result combiner 140 maycalculate a score for BI-RADS Category 3 to be 0.2, which is a sum of0.1 (a frame score of the image frame 2) and 0.1 (a frame score of theframe image 9). The result combiner 140 may calculate a score forBI-RADS Category 4A to be 1.8 which is a sum of 0.3 (a frame score ofthe image frame 3), 0.5 (a frame score of the image frame 4), 0.5 (aframe score of the frame image 5), and 0.5 (a frame score of the imageframe 6). The result combiner 140 may calculate a score for BI-RADSCategory 4B to be 0.8, which is a sum of 0.5 (a frame score of the imageframe 7) and 0.3 (a frame score of the image frame 8).

In an example, a frame score may be calculated by Equation 1 that isrepresented as below:

FS _(n)=Fitness_(n) ×NLS _(n)   [Equation 1]

In Equation 1, n denotes a numeric value of an image frame, denotes anormalized size of an ROI detected from the n^(th) image frame (e.g., arelative size of an ROI detected in the n^(th) image frame withreference to the largest size of an ROI detected from an image framegroup, or a normalized size of lesion), and Fitness_(n) denotes modelfitness of the n^(th) image frame.

The model fitness may be a probability. For example, a lesionclassification result regarding an image frame is given in a form ofprobability distribution. When the classification shows BI-RADS Category1, 2, 3, 4A, 4B, and 5, only BI-RADS Category 1 may used as a finalclassification result if a probability of the image frame 1 to beBI-RADS Category 1 is 75%(0.75) and a probability to be BI-RADS Category2 is 25%(0.25). To calculate a frame score (FS), probabilities regardingall the categories may be multiplexed.

Based on a score for each RADS Lexicon or BI-RADS Category item in agroup, the result combiner 140 may generate a histogram of the imageframe group, such as, for example, a Lexicon histogram or a Categoryhistogram, as a group classification result for the image frame group.

The result combiner 140 may combine all classification results for imageframes belonging to a group to determine whether an ROI of the group ismalignant or benign in an attempt to calculate a group classificationresult for the group. For example, the result combiner 140 may determinewhether an ROI of a group is malignant or benign based on a BI-RADSLexicon item with the highest score or a BI-RADS Category item with thehighest score in the group.

FIG. 2 is a diagram illustrating another example of a CAD apparatus.Referring to FIG. 2, a CAD apparatus 200 may include a similar imagecombiner 210, a similar group combiner 220, and a display component 230,in addition to the configurations of the CAD apparatus 100 shown inFIG. 1. Many of the components shown in FIG. 2 have been described withreference to FIG. 1. The above description of FIG. 1, is also applicableto FIG. 2, and is incorporated herein by reference. Thus, the abovedescription may not be repeated here.

The similar image combiner 210 may combine similar image frames thatsatisfy a predetermined condition among continuous image frames, andselect any image frame of the combined similar image frames as arepresentative image frame. The predetermined condition may include acondition where a change in the size of an ROI needs to be smaller thana predetermined threshold and a condition where the maximum size of anROI needs to be smaller than a predetermined threshold.

In this case, when combining all classification results, the resultcombiner 140 may use only the representative image frame among thecombined similar image frames.

If a change in the size of an ROI or a maximum size of an ROI is toosmall, the ROI may be insignificant. Thus, by combining image frameswith the insignificant ROI into a single image frame, it is possible toimprove performance.

The similar group combiner 220 may combine similar groups into onegroup. Specifically, the similar group combiner 220 may generate ahistogram representing characteristics of an ROI of each group, andcombine similar groups into one group by using a histogram clusteringalgorithm. The characteristics of an ROI may be Lexicon information onthe ROI, such as, for example, Round, Oval, Lobular, Microlobulated(irregular), Circumscribe, Indistinct, Spiculated, Fat containing,Density Low, Density Equal, and Density high.

For example, the similar group combiner 220 may combine similar groupsinto one group by using a Hierarchical Agglomerative Clusteringalgorithm, such as, for example, Hierarchical Agglomerative Clusteringalgorithm and a Bhattacharyya Coefficient algorithm.

As described above, the grouping component 120 may group image frameswith the same ROI into groups based on whether the ROI is continuouslydetected. In this case, the same ROI may be classified redundantly andclassification results thereof may be inconsistent. Thus, by combiningsimilar groups into one group, it is possible to improve performance ofthe system.

The display component 230 may display a group classification result foreach group on a screen. For example, the display component 230 maydisplay information on a BI-RADS Lexicon item with the highest score ora BI-RADS Category item with the highest score in each group, a Lexiconor Category histogram of each group, and a determination as to whetheran ROI of each group is malignant or benign.

FIGS. 3A and 3B are diagrams illustrating examples of operations of aCAD apparatus.

Referring to FIGS. 3A and 3B, a user moves a probe 320 to the right andthen to the left. In this case, with respect to a patient's specificbody part 330, the CAD apparatus 200 acquires image frames 1 to 7 (301to 307) sequentially when moving the probe 320 to the right, and imageframes 8 to 14 (308 to 314) sequentially when moving the probe 320 tothe left.

Once the image frames 1 to 14 (301 to 314) are acquired, the ROIdetector 110 may detect an ROI 340 from each of the image frames 1 to 14(301 to 314). An ROI is detected from the image 2 to 7 (302 to 307) andthe image frames 9 to 14 (309 to 314), but no ROI is detected from theimages 1 and 8.

With reference to the image frame 1 (301) and the image frame 8 (308),the grouping component 120 may group the image frames 2 to 7 (302 to307) and the image frames 9 to 14 (309 to 314) into two differentgroups. The grouping component 120 may group the image frames 2 to 7(302 to 307) into group A, and the image frames 9 to 14 (309 to 314)into group B.

With respect to image frames (302 to 307 and 309 to 314) of each group(group A and group B), the classifier 130 may classify an ROI detectedfrom each image frame. In FIGS. 3A and 3B, image frame 2 (302) and 14(314) are classified as BI-RADS Category 2; image frames 7 (307) 9 (309)are classified as BI-RADS Category 3; image frames 3 (303), 4 (304), 6(306), 10 (310) and 13 (313) are classified as BI-RADS Category 4A; andimage frames 5 (305), 11 (311) and 12 (312) are classified as BI-RADSCategory 4B.

The similar group combiner 220 may combine similar groups, such as groupA and group B, into one group. In FIGS. 3A and 3B, image frames (302 to307) in group A and image frames (309 to 314) in group B are imagescapturing the same ROI. Characteristics of the ROI of group A andcharacteristics of the ROI of group B are similar, and thus, the similargroup combiner 220 combines group A and group B into one group.

The result combiner 140 may calculate a score for each BI-RADS Categoryitem in each group based on BI-RADS Category information of each imageframe (302 to 307 and 309 to 314) in the combined group, and generate ahistogram, as shown in FIG. 3B, based on the calculated scores. In FIGS.3A and 3B, a score for BI-RADS Category 2 and BI-RADS Category 3 iscalculated to be 2, a score for Category 4A is calculated to be 5, and ascore for Category 4B is calculated to be 3. Based on the calculatedscores, the result combiner 140 generates a Category histogram 350 ofthe combined group.

FIG. 4 is a diagram illustrating an example of a group classificationresult displayed on a screen. Referring to FIG. 4, as a groupclassification result for each group, the display component 230 maydisplay, on the screen 410, a Lexicon histogram 424 of each group, aCategory histogram 422 of each group, and a determination 426 as towhether an ROI of each group is malignant or benign.

The information displayed by the display component 230 on the screen 410are non-exhaustive illustration, and other arrangements and types ofinformation are considered to be well within the scope of the presentdisclosure. For example, the display component 230 may display, on thescreen 410, information on a BI-RADS Lexicon item with the highest scoreor a BI-RADS Category item with the highest score in each group.

FIG. 5 is a diagram illustrating an example of a CAD method. Theoperations in FIG. 5 may be performed in the sequence and manner asshown, although the order of some operations may be changed or some ofthe operations omitted without departing from the spirit and scope ofthe illustrative examples described. Many of the operations shown inFIG. 5 may be performed in parallel or concurrently. The abovedescription of FIGS. 1-4, is also applicable to FIG. 5, and isincorporated herein by reference. Thus, the above description may not berepeated here.

In 510, a CAD method 500 includes detecting ROIs from a plurality ofimage frames. For example, the CAD apparatus 100 may detect ROIs from aplurality of image frames using a lesion detection algorithm such as,for example, AdaBoost, DPM, DNN, CNN, and Sparse Coding.

In 520, a plurality of image frames having successive ROI sections aregrouped into one group based on the result of the ROI detection. Forexample, the CAD apparatus 100 may group a plurality of image framesinto groups with reference to an image frame in which no ROI isdetected. In this case, even image frames from which the same ROI isdetected may be grouped into different groups according to whether theROI is continuously detected.

In 530, an ROI detected from each image frame of each group isclassified. For example, the CAD apparatus 100 may extract a featurevalue from an ROI detected from each image frame, and classify thedetected ROI based on the extracted feature value. The classificationresult regarding the ROI may include information such as, for example,BI-RADS Lexicon information on the ROI, BI-RADS Category information onthe ROI, and whether the ROI is malignant/benign.

In 540, the CAD apparatus 100 calculates a group classification resultfor a group by combining all classification results for image framesbelonging to the group.

For example, the CAD apparatus 100 may calculate a score for eachBI-RADS Lexicon item or each BI-RADS Category item in a group based onBI-RADS Lexicon information or BI-RADS Category information on an ROIdetected from each image frame in the group. The CAD apparatus 100 mayselect a BI-RADS Lexicon item with the highest score or a BI-RADSCategory item with the highest score as a group classification resultfor the group. The score for each item may be obtained by calculatingthe number of image frames corresponding to each BI-RADS Lexicon item oreach BI-RADS Category item or by adding up frame scores of image framescorresponding to each BI-RADS Lexicon item or each BI-RADS Categoryitem. A frame score may be calculated by Equation 1 above.

In another example, the CAD apparatus 100 may generate a histogram(Lexicon histogram or Category histogram) of a group as a groupclassification result for the group based on a score for each BI-RADSLexicon item or each BI-RADS Category item in the group.

In yet another example, the CAD apparatus 100 may combine allclassification results for image frames belonging to the group todetermine whether an ROI of a group is malignant or benign in an effortto calculate a group classification result for the group.

FIG. 6 is a diagram illustrating another example of a CAD method. Theoperations in FIG. 6 may be performed in the sequence and manner asshown, although the order of some operations may be changed or some ofthe operations omitted without departing from the spirit and scope ofthe illustrative examples described. Many of the operations shown inFIG. 6 may be performed in parallel or concurrently. The abovedescription of FIGS. 1-5, is also applicable to FIG. 6, and isincorporated herein by reference. Thus, the above description may not berepeated here.

Referring to FIG. 6, a CAD method 600 may further include combiningsimilar images in 610, combining similar groups in 620, and displaying aresult on a screen in 630, in addition to operations shown in the CADmethod 500.

In 610, similar image frames satisfying a predetermined condition fromamong continuous image frames are combined. For example, the CADapparatus 200 combines similar image frames satisfying a predeterminedcondition from among continuous image frames, and selects any imageframe from among the combined similar image frames as a representativeimage frame. The predetermined condition may include conditions such as,for example, a condition where a change in the size of an ROI needs tobe smaller than a predetermined threshold and a condition where themaximum size of an ROI needs to be smaller than a predeterminedthreshold.

If a change in the size of an ROI or the maximum size of an ROI is toosmall, the ROI may be insignificant. Thus, by combining all the imageframes including the insignificant ROI into one image frame, it ispossible to improve performance of the system.

In 620, similar groups are combined into one group. For example, the CADapparatus 200 may generate a histogram representing characteristics ofan ROI of each group, and combine similar groups into one group using ahistogram clustering algorithm, such as, for example, a HierarchicalAgglomerative Clustering algorithm, and a Bhattacharyya Coefficientalgorithm. The characteristics of an ROI may be Lexicon information ofthe ROI, such as, for example, Round, Oval, Lobular, Microlobulated(irregular), Circumscribe, Indistinct, Spiculated, Fat containing,Density Low, Density Equal, and Density high.

As described above regarding operation 520, even image frames from whichthe same ROI is detected may be classified into different groupsaccording to whether the ROI is continuously detected. In this case, thesame ROI may be classified redundantly and classification resultsthereof may be inconsistent. Thus, by combining similar groups into onegroup, it is possible to improve performance of the system.

In 630, a group classification result for each group may be displayed ona screen. For example, the CAD apparatus 200 may display, on a screen,information such as, for example, a BI-RADS Lexicon item with thehighest score or a BI-RADS Category item with the highest score in eachgroup, Lexicon histogram or Category histogram for each group, andwhether an ROI of each group is malignant/benign.

The apparatuses, units, modules, devices, and other componentsillustrated that perform the operations described herein are implementedby hardware components. Examples of hardware components includecontrollers, sensors, generators, drivers and any other electroniccomponents known to one of ordinary skill in the art. In one example,the hardware components are implemented by one or more processors orcomputers. A processor or computer is implemented by one or moreprocessing elements, such as an array of logic gates, a controller andan arithmetic logic unit, a digital signal processor, a microcomputer, aprogrammable logic controller, a field-programmable gate array (FPGA), aprogrammable logic array, a microprocessor, an application-specificintegrated circuit (ASIC), or any other device or combination of devicesknown to one of ordinary skill in the art that is capable of respondingto and executing instructions in a defined manner to achieve a desiredresult. In one example, a processor or computer includes, or isconnected to, one or more memories storing instructions or software thatare executed by the processor or computer. Hardware componentsimplemented by a processor or computer execute instructions or software,such as an operating system (OS) and one or more software applicationsthat run on the OS, to perform the operations described herein. Thehardware components also access, manipulate, process, create, and storedata in response to execution of the instructions or software. Forsimplicity, the singular term “processor” or “computer” may be used inthe description of the examples described herein, but in other examplesmultiple processors or computers are used, or a processor or computerincludes multiple processing elements, or multiple types of processingelements, or both. In one example, a hardware component includesmultiple processors, and in another example, a hardware componentincludes a processor and a controller. A hardware component has any oneor more of different processing configurations, examples of whichinclude a single processor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. 5-6 that perform the operationsdescribed herein are performed by a processor or a computer as describedabove executing instructions or software to perform the operationsdescribed herein.

Instructions or software to control a processor or computer to implementthe hardware components and perform the methods as described above arewritten as computer programs, code segments, instructions or anycombination thereof, for individually or collectively instructing orconfiguring the processor or computer to operate as a machine orspecial-purpose computer to perform the operations performed by thehardware components and the methods as described above. In one example,the instructions or software include machine code that is directlyexecuted by the processor or computer, such as machine code produced bya compiler. In another example, the instructions or software includehigher-level code that is executed by the processor or computer using aninterpreter. Programmers of ordinary skill in the art can readily writethe instructions or software based on the block diagrams and the flowcharts illustrated in the drawings and the corresponding descriptions inthe specification, which disclose algorithms for performing theoperations performed by the hardware components and the methods asdescribed above.

The instructions or software to control a processor or computer toimplement the hardware components and perform the methods as describedabove, and any associated data, data files, and data structures, arerecorded, stored, or fixed in or on one or more non-transitorycomputer-readable storage media. Examples of a non-transitorycomputer-readable storage medium include read-only memory (ROM),random-access memory (RAM), flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs,CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs,BD-Rs, BD-R LTHs, BD-REs, magnetic tapes, floppy disks, magneto-opticaldata storage devices, optical data storage devices, hard disks,solid-state disks, and any device known to one of ordinary skill in theart that is capable of storing the instructions or software and anyassociated data, data files, and data structures in a non-transitorymanner and providing the instructions or software and any associateddata, data files, and data structures to a processor or computer so thatthe processor or computer can execute the instructions. In one example,the instructions or software and any associated data, data files, anddata structures are distributed over network-coupled computer systems sothat the instructions and software and any associated data, data files,and data structures are stored, accessed, and executed in a distributedfashion by the processor or computer.

While this disclosure includes specific examples, it will be apparent toone of ordinary skill in the art that various changes in form anddetails may be made in these examples without departing from the spiritand scope of the claims and their equivalents. The examples describedherein are to be considered in a descriptive sense only, and not forpurposes of limitation. Descriptions of features or aspects in eachexample are to be considered as being applicable to similar features oraspects in other examples. Suitable results may be achieved if thedescribed techniques are performed in a different order, and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents. Therefore, the scope of the disclosureis defined not by the detailed description, but by the claims and theirequivalents, and all variations within the scope of the claims and theirequivalents are to be construed as being included in the disclosure.

What is claimed is:
 1. A Computer Aided Diagnosis (CAD) apparatuscomprising: a display; a memory configured to store instructions; and atleast one processor, upon executing the stored instructions, configuredto: detect a Region of Interest (ROI) from a plurality of image frames,generate at least one group of image frames based on a result of thedetection, wherein the at least one group comprises image frames inwhich the ROI is detected, from among the plurality of image frames,extract a value from at least one of a contour of the ROI comprised inthe image frames, an area adjacent to the contour, or pixels inside thecontour, classify the ROI comprised in each of the image framesbelonging to the at least one group by comparing the extracted valuewith a pre-stored criteria, determine a group result for the each groupbased on a plurality of classification results for the image framesbelonging to the each group, and control the display to display thegroup result on a screen.
 2. The CAD apparatus of claim 1, wherein theat least one processor is further configured to group a plurality ofsuccessive image frames into a group based on an image frame where noROI is detected.
 3. The CAD apparatus of claim 1, wherein theclassification result comprises at least one of BI-RADS Lexiconinformation or BI-RADS Category information on a corresponding ROI, andwherein the at least one processor is further configured to: calculate ascore for each BI-RADS Lexicon item or each BI-RADS Category item in theeach group based on the classification results, and select a BI-RADSLexicon item with the highest score or a BI-RADS Category item with thehighest score as the group result.
 4. The CAD apparatus of claim 3,wherein the at least one processor is further configured to: calculatethe score for each BI-RADS Lexicon item or each BI-RADS Category item inthe each group by adding up frame scores of image frames correspondingto each BI-RADS Lexicon item or each BI-RADS Category item in the group,and wherein the frame scores represent weights assigned to each of theimage frames.
 5. The CAD apparatus of claim 4, wherein the frame scoreis calculated based on at least one of a size of an ROI detected fromeach of the image frames in the each group and a model fitness of theeach of the image frames in the each group.
 6. The CAD apparatus ofclaim 3, wherein the at least one processor is further configured togenerate a histogram of the group based on the calculated score for eachBI-RADS Lexicon item or each BI-RADS Category item in the each group. 7.The CAD apparatus of claim 3, wherein the at least one processor isfurther configured to determine whether an ROI of the each group isbenign based on the BI-RADS Lexicon item with the highest score or theBI-RADS Category item with the highest score in the each group.
 8. TheCAD apparatus of claim 1, wherein the at least one processor is furtherconfigured to: combine image frames satisfying a condition from amongthe image frames, select any one of the combined images as arepresentative image frame, and use the representative image frame whencombining the classification results.
 9. The CAD apparatus of claim 8,wherein the condition comprises at least one of a condition where achange in size of an ROI is smaller than a threshold value or acondition where a maximum size of an ROI is smaller than a thresholdsize.
 10. The CAD apparatus of claim 1, wherein the at least oneprocessor is further configured to: generate a histogram representingcharacteristics of an ROI of the each group, and combine similar groupsinto one group using a histogram clustering algorithm.
 11. The CADapparatus of claim 10, wherein the histogram clustering algorithmcomprises one of a Hierarchical Agglomerative Clustering algorithm and aBhattacharyya Coefficient algorithm.
 12. A Computer Aided Diagnosis(CAD) method comprising: detecting a Region of Interest (ROI) from aplurality of image frames; grouping image frames in which the ROI isdetected, from among the plurality of image frames based on a result ofthe detection; extracting a value from at least one of a contour of theROI comprised in the image frames, an area adjacent to the contour, orpixels inside the contour; classifying the ROI comprised in each of theimage frames belonging to a group by comparing the extracted value witha pre-stored criteria; determining a group result for the group based ona plurality of classification results for the image frames belonging tothe group; and displaying the group result on a display.
 13. The CADmethod of claim 12, wherein the grouping of the plurality of imageframes comprises grouping a plurality of successive image frames withreference to an image frame where no ROI is detected.
 14. The CAD methodof claim 12, wherein the classifying of the ROI comprises identifying atleast one of BI-RADS Lexicon information or BI-RADS Category informationon the ROI comprised in each of the image frames, and wherein thedetermining of the group result comprises: calculating a score for eachBI-RADS Lexicon item or each BI-RADS Category item in the successiveimage frames belonging to the group; and selecting a BI-RADS Lexiconitem with the highest score or BI-RADS Category item with the highestscore in the group as the group result.
 15. The CAD method of claim 14,wherein the calculating of the score for each BI-RADS Lexicon item oreach BI-RADS Category item comprises: calculating a frame score based onat least one of a size of an ROI comprised in each of the image framesand a model fitness of each of the image frames; and calculating thescore for each BI-RADS Lexicon item or each BI-RADS Category item in thegroup by adding up frame scores for image frames corresponding to eachBI-RADS Lexicon item or each BI-RADS Category item in the group.
 16. TheCAD method of claim 14, further comprising generating a histogram of thegroup based on the calculated score for each BI-RADS Lexicon item oreach BI-RADS Category item in the group.
 17. The CAD method of claim 12,further comprising: combining image frames that satisfy a condition fromamong the successive image frames, and selecting any one of the combinedimage frames as a representative image frame; and using therepresentative image frame in combining the classification results forthe successive image frames belonging to the group, wherein thecondition comprises at least one of a condition where a change in sizeof an ROI is less than a threshold value or a condition where a maximumsize of an ROI is less than a threshold size.
 18. The CAD method ofclaim 12, further comprising: generating a histogram representingcharacteristics of an ROI of each group; and combining similar groupsinto one group using a histogram clustering algorithm.
 19. The CADmethod of claim 15, wherein the model fitness of the each of the imageframes corresponds to a probability distribution of the each BI-RADSLexicon item or each BI-RADS Category item.
 20. A computer programproduct comprising a non-transitory computer-readable storage mediumconfigured to store a computer readable program comprising instructionsconfigured to, when executed by a computing device, cause the computingdevice to: detect a Region of Interest (ROI) from a plurality of imageframes; generate at least one group of image frames based on a result ofthe detection, wherein the at least one group comprises image frames inwhich the ROI is detected, from among the plurality of image frames;extract a value from at least one of a contour of the ROI comprised inthe image frames, an area adjacent to the contour, or pixels inside thecontour; classify the ROI comprised in each of the image framesbelonging to the at least one group by comparing the extracted valuewith a pre-stored criteria; and determine a group result for the eachgroup based on a plurality of classification results for the imageframes belonging to the each group.